Section: New Results

Robust Remote Heart Rate Estimation from Face Utilizing Spatial-temporal Attention

Participants : Antitza Dantcheva, Abhijit Das, Xuesong Niu [Chinese Academy of Sciences] , Xingyuan Zhao [Chinese Academy of Sciences] , Hu Han [Chinese Academy of Sciences] , Shiguang Shan [Chinese Academy of Sciences] , Xilin Chen [Chinese Academy of Sciences] .

We proposed an end-to-end approach for robust remote heart rate (HR) measurement gleaned from facial videos. Specifically the approach was based on remote photoplethysmography (rPPG), which constitutes a pulse triggered perceivable chromatic variation, sensed in RGB-face videos. Incidentally rPPGs can be affected in less-constrained settings. To unpin the shortcoming, the proposed algorithm utilized a spatio-temporal attention mechanism, which placed emphasis on the salient features included in rPPG-signals. In addition, we proposed an effective rPPG augmentation approach, generating multiple rPPG signals with varying HRs from a single face video. Experimental results on the public datasets VIPL-HR and MMSE-HR showed that the proposed method outperformed state-of-the-art algorithms in remote HR estimation. This work has been presented at the IEEE International Conference on Automatic Face and Gesture Recognition (FG 2019) [28].

Figure 14. Overview of the proposed end-to-end trainable approach for rPPG based remote HR measurement via representation learning with spatial-temporal attention.